Bug NTLM authentication wrong behavior with the head flag --head or -I

[Mystic-Ivy]

I did this

Trying to authenticate against an NTLM endpoint with the -I flag. Here the command seems to forget the received header from the server.

$ curl -I --ntlm --user [REDACTED]\[REDACTED]:[REDACTED] http://<IP> --trace-ascii /dev/out
== Info:   Trying[REDACTED]:80...
== Info: Connected to [REDACTED] ([REDACTED]) port 80
== Info: Server auth using NTLM with user '[REDACTED]'
=> Send header, 143 bytes (0x8f)
0000: HEAD / HTTP/1.z
0011: Host: [REDACTED]
0026: Authorization: NTLM [REDACTED]
0068: User-Agent: curl/8.7.1
0080: Accept: */*
008d:
== Info: Request completely sent off
<= Recv header, 27 bytes (0x1b)
0000: HTTP/1.1 401 Unauthorized
HTTP/1.1 401 Unauthorized
<= Recv header, 25 bytes (0x19)
0000: Content-Type: text/html
Content-Type: text/html
<= Recv header, 27 bytes (0x1b)
0000: Server: Microsoft-IIS/8.5
Server: Microsoft-IIS/8.5
<= Recv header, 289 bytes (0x121)
0000: WWW-Authenticate: NTLM [REDACTED]
0040: [REDACTED]
0080: [REDACTED]
00c0: [REDACTED]
0100: [REDACTED]
WWW-Authenticate: NTLM [REDACTED]
<= Recv header, 29 bytes (0x1d)
0000: WWW-Authenticate: Negotiate
WWW-Authenticate: Negotiate
<= Recv header, 37 bytes (0x25)
0000: Date: Mon, 22 Mar 2021 20:41:54 GMT
Date: Mon, 22 Mar 2021 20:41:54 GMT
<= Recv header, 19 bytes (0x13)
0000: Content-Length: 5
Content-Length: 5
<= Recv header, 2 bytes (0x2)
0000:

== Info: Ignoring the response-body
== Info: Closing connection
== Info: Issue another request to this URL: 'http://[REDACTED]/'
== Info: Hostname [REDACTED] was found in DNS cache
== Info:   Trying [REDACTED]:80...
== Info: Connected to [REDACTED] ([REDACTED]) port 80
== Info: Server auth using NTLM with user '[REDACTED]'
=> Send header, 143 bytes (0x8f)
0000: HEAD / HTTP/1.1
0011: Host: [REDACTED]
0026: Authorization: NTLM [REDACTED]
0068: User-Agent: curl/8.7.1
0080: Accept: */*
008d:
== Info: Request completely sent off
<= Recv header, 27 bytes (0x1b)
0000: HTTP/1.1 401 Unauthorized
HTTP/1.1 401 Unauthorized
<= Recv header, 25 bytes (0x19)
0000: Content-Type: text/html
Content-Type: text/html
<= Recv header, 27 bytes (0x1b)
0000: Server: Microsoft-IIS/8.5
Server: Microsoft-IIS/8.5
<= Recv header, 289 bytes (0x121)
0000: WWW-Authenticate: NTLM [REDACTED]
0040: [REDACTED]
0080: [REDACTED]
00c0: [REDACTED]
0100: [REDACTED]
WWW-Authenticate: NTLM [REDACTED]
<= Recv header, 29 bytes (0x1d)
0000: WWW-Authenticate: Negotiate
WWW-Authenticate: Negotiate
<= Recv header, 37 bytes (0x25)
0000: Date: Mon, 22 Mar 2021 20:41:54 GMT
Date: Mon, 22 Mar 2021 20:41:54 GMT
<= Recv header, 19 bytes (0x13)
0000: Content-Length: 5
Content-Length: 5
<= Recv header, 2 bytes (0x2)
0000:

I expected the following

When explicitly using the request flag and setting it to HEAD the authentication works as expected:

$ curl -X HEAD --ntlm --user [REDACTED]\[REDACTED]:[REDACTED] http://[REDACTED] --trace-ascii /dev/out
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the
Warning: way you want. Consider using -I/--head instead.
== Info:   Trying [REDACTED]:80...
== Info: Connected to [REDACTED] ([REDACTED]) port 80
== Info: Server auth using NTLM with user '[REDACTED]'
=> Send header, 143 bytes (0x8f)
0000: HEAD / HTTP/1.1
0011: Host: [REDACTED]
0026: Authorization: NTLM [REDACTED]
0068: User-Agent: curl/8.7.1
0080: Accept: */*
008d:
== Info: Request completely sent off
<= Recv header, 27 bytes (0x1b)
0000: HTTP/1.1 401 Unauthorized
<= Recv header, 25 bytes (0x19)
0000: Content-Type: text/html
<= Recv header, 27 bytes (0x1b)
0000: Server: Microsoft-IIS/8.5
<= Recv header, 289 bytes (0x121)
0000: WWW-Authenticate: NTLM [REDACTED]
0040: [REDACTED]
0080: [REDACTED]
00c0: [REDACTED]
0100: [REDACTED]
<= Recv header, 29 bytes (0x1d)
0000: WWW-Authenticate: Negotiate
<= Recv header, 37 bytes (0x25)
0000: Date: Mon, 22 Mar 2021 20:41:54 GMT
<= Recv header, 19 bytes (0x13)
0000: Content-Length: 5
<= Recv header, 2 bytes (0x2)
0000:
== Info: Ignoring the response-body
== Info: Connection #0 to host [REDACTED] left intact
== Info: Issue another request to this URL: 'http://[REDACTED]/'
== Info: Found bundle for host: 0x5f8a1c748ec0 [serially]
== Info: Can not multiplex, even if we wanted to
== Info: Re-using existing connection with host [REDACTED]
== Info: Server auth using NTLM with user '[REDACTED]'
=> Send header, 499 bytes (0x1f3)
0000: HEAD / HTTP/1.1
0011: Host: [REDACTED]
0026: Authorization: NTLM [REDACTED]
0066: [REDACTED]
00a6: [REDACTED]
00e6: [REDACTED]
0126: [REDACTED]
0166: [REDACTED]
01a6: [REDACTED]
01cc: User-Agent: curl/8.7.1
01e4: Accept: */*
01f1:
== Info: Request completely sent off

curl/libcurl version

curl 8.7.1 (x86_64-pc-linux-gnu) libcurl/8.7.1 OpenSSL/3.3.0 zlib/1.3.1 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 nghttp3/1.2.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

operating system

Arch Linux

[bagder]

If you add --trace-config write to the command line you will get added tracing information for the writing of download data.

I suspect that trace output will show that the HEAD requests actually get body data back and that curl consider that an error (it is a protocol violation) and marks the connection for close. Since NTLM authenticates connections, the next request then starts a new connection and the NTLM auth has to start over again...

[bagder]

No response